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ABSTRACT 

A Study assessed the reliability and some aspects 
related to the validity of an open sentence approach to assessing 
comprehension during silent reading. Subjects, 37 randomly selected 
seventh-grade students, read an *'Opin" passage, prepared by deleting 
the second half of every other sentence. The blanks filled in by 
students were evaluated using a scale based on the response's 
resemblance to the author's intent. Responses were also divided into 
fragmented, projective, and interactive categories. Oral readings of 
a second, intact passage were used to determine the kinds of reading 
miscues made by students. The results confirm that the Opin sentence 
procedure has sufficient interrater reliability to be used for 
assessing a reader's ability to interact with text during silent 
reading. Both the percent score method and the category scoring 
method proved reliable for reading assessment purposes. The pattern 
€»f correlations with the miscue scores also indicates that readers 
who attempt to interact with the text while reading are more likely 
to make miscues that are consistent with their search for meaning. 
(Tables of data and a sample Opin passage are appended.) (HTH) 
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Assessing Silent Reading During the Process: 
An Investigation of the Opin Procedure 
Modern cognitive theorists (Rumelhart, 1981, Rumelhart & 
Ortony, 1977) have described reading comprehension as an active 
process which requires the application of prior knowledge. 
According to these theorists our knowledge is represented in 
abstract units called schemata. These schemes are thought to 
play a central role in the process of comprehension. During 
reading active schemes influence anticipatory and inferential 
thinking by providing the "empty slots" onto which the reader 
maps the elements of the text (Anderson, 1977) . Goodman 
(1967/1976) describes this process as guessing (a 
psycholinguistic guessing game). The reader anticipates both 
the author's ideas and the language by which the author may 
convey those ideas symbolized in print. Thus, schemes enable 
the reader to set-up expectations based upon prior knowledge 
that are either confirmed modified, or disconfirmed as the 
reader encounters the actual details of the passage. It would 
seem that the "better" reader makes better guesses at both the 
ideas and the language of the aut hor . 

Based on this view of reedi.Ag con.prehension it would appear 
logical to differentiate good fron poor comprehenders by 
examining the process of hypothesis generation and evaluation 
during the process of comprehension (Rumelhart, 1984). However, 
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there are very few assessment instruments now available which 
allow reading researchers, diagnosticians and teachers to view 
the reading process during the act of reading. The reading 
miscue inventory developed by Goodman and Burke (1972) and the 
cloze procedure developed are two such process oriented 
assessment devices. The miscue analysis identifies which clues 
the readers relied on most while reading and thus provides 
valuable information about the identification processes the 
reader uses as well as^ some indirect evidence of the reader's 
comprehension activities.. However, miscue analysis cannot be 
used tc assess the silent reading process. The cloze procedure 
provides an assessment of the readers use of local 
contextual/linguistic clues in word identification and sent-^nce 
comprehension but does not appear to adequately assess 
intersentence comprehension (Shanahan, Kamil, and Tobin, 1932). 
Thus it is apparent that neither one of these assessment 
procedures adequately captures the act of comprehending as it 
has been described by current theorists. Consequently, there is 
a need for tests which involve assessment of the comprehension 
process during the act of reading (Rumelhart, 1984). 

In investigating reading comprehension, the interaction 
between the reader and writer would seem to be of paramount 
importance because it lies at the heart of the communication 
process. The reader brings to the page his or her organized 
emotional, conceptual, and linguistic experiences. Based on 
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experiences, the author represents his or her intended meanings, 
in the form of a written code. The reader is more likely to be 
successful in constructing meanings which are related to the 
author's if there is a commonality of experiences* This 
relationship between the author's and reader's experiences can 
be described as proximity (Pehrsson & Robinson, 1985). When 
there is a high degree of proximity between the author and 
reader, the reader will be more likely to make guesses based on 
their prior knowledge schemes that are confirmed by the passage. 
The less proximity, the more the reader and the author will 
diverge and thus, the greater the difficulty that the reader 
will have in comprehending the intended message of the writer 
(Pehrsson and Robinson, 1985) . 

If the reader is able to interact with the author 
successfully then the reader may even be able to predict the 
surface structure of the author's message. Surface structure is 
the observable sentence, written or spoken and the relationship 
among its words (Wardhaugh, 1975). Page (1974) suggested that 
the conceived surface structure is the author's internal concept 
of the sentence about to be written. Page also defined the 
perceived surface structure as the reader's internalized 
perception of the observable surface structure. Both perceived 
and conceived structure are unobservable. According to both the 
theories of Goodman (1967/1976) and Rumelhart (1984), when the 
reader and author interact the conceived surface structure of 
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the writer can be predicted by the reader. 

The prediction by the reader could be analyzed in terms of 
the degree of match obtained with the author's conceived surface 
structure. One way to evaluate the reader's ability to guess 
the intended meaning of the author would be to have them attempt 
to finish the author's sentences for him. This could be 
achieved by deleting half of every other sentence in a short 
passage anc asking the reader to complete the open sentences 
using their opinion (hypothesis) as to what the author might 
have written in the blank. Under this procedure , the reader is 
asked to guess the surface structure tnat the author has written 
based on his or her ability to activate relevant knowledge 
schemes similar to those intended by the author and based on 
clues from the remaining surface structure. We have chosen to 
call this technique the Opin procedure because of the open 
nature of the sentence deletions and the fact that the reader 
must develop his or her own opinion as to the conceived 
structure the writer intended. 

The purpose of this study is to assess the reliability and 
some aspects related to the validity of an open sentence 
approach to assessinq comprehension during the act of silent 
reading. It is expected that the procedure will display 
sufficient reliability to b€ useful for reading assessment 
purposes. It is predicted ^ however r that Opin procedure 
comprehension scores will show only moderate correlations with 
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standardized reading achievement test scores that emphasize 
•'product'' rather than "process". Finally, it is expected that 
performance on the Opin procedure will correlate with 
performance patterns based on a miscue analysis, which is 
another process measure. It is anticipated that students who 
are able to interact with the writer will demonstrate a greater 
propensity while reading to make miscue error that both make 
sense and preserve the author's intent than readers who show 
less ability to interact with the text. 

METHODS 

Subjects 

A total of 37 seventh-grade junior high school students 
from Idaho participated in the study. The subjects were 
randomly selected from a group of 178 seventh-grade students in 
advanced, developmental and remedial reading classes. 
Materials 

The passage used for the Opin sentence procedure was titled 
"Moon May Change Way People Behave" ( You and Your World. , 
Xerox, 1974). The article describes the possibility that the 
moon affects human behavior. Tho readability level of the 
passage as computed by the Dale-Chall (1954) and Fry (1978) 
formulas fell in the sixth-seventh grade range. This passage 
was prepared for use with the Opin procedure by deleting the 
second half of every other sentence, which resulted in 12 Opin 
deletions for the experimental passage. In place of the deleted 
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sentence ending a line was drawn approximately equal to the 
length of the page. The title of the passage was also deleted. 
As an illustration, a copy of the passage used in the study 
including the Opin sentence deletions is presented in Figure 1* 
The information written in the blanks is the actual words used 
by the author to complete the sentences. 

A second passage at the same reading grade-level range was 
used to record a sample of the kinds of reading miscues made by 
the students. The passage was titled , ••Earthquakes" (Brown & 
Kemper, 1982) . 
Scor ing 

For the Opin procedure, the blanks filled-in by the 
students were evaluated separately using the following five 
point scale: 

4 - same or very similar to the author's intent. 
3 - different, but contributed to the author's intent. 
2 = different, failed to contribute to the author's intent, 
but related to a topic supplied by the reader vhich 
was used to make sense out of the passage. 
1 s made sense only in the local context and failed to 

contribute to an intersentence topic. 
0 - Failed to make any sense or was left blank. 
A total score was computed for «ach student by adding up 
the number of points they received for each of the Opin 
deletions. Because the number of Opin blanks will vary with the 
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length of the reading selection (or sample length) r the Opin 
score was computed by dividing the total raw score by the number 
of Opin deletions. The result was a percentage score which 
reflected the ability of the reader to interact with the text. 
The higher the score r the closer the proximity between the 
reader and the author . 

A second means of scoring the responses to the Opin 
procedure was to count the number of responses in each of three 
categories: (1) Fragmented y responses that make sense out of the 
passage only within the local context , if at all, and without 
reference to an integrating scheme , (2) Projective^ responses 
that were integrated around an hypothesized scheme but without 
reference to the scheme intended by the author ^ and (3) 
Interactive^ responses that accurately reflected the scheme 
intended fcy the author. Students were classified according to 
the above described category system whenever 50% or more of 
their responses fall within one of these categories. 

For the miscue analysis, the rater used a copy of the 
earthquakes passage to record the miscues made by each student. 
The rater listened to a tape recording of the student's oral 
reading, and marked all variations between the expected 
responses and the student's actual responses. Counted miscues 
included all substitutions, omissions, insertions, reversals, 
and repetitions. Each miscue was coded according to the level 
of the clues the reader appeared to rely upon and the probable 
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source of the miscue* The classifications used to code the 
miscues were: (1) makes sense, (2) same meaning, (3) syntactic 
fit, (4) syntactic fit up to miscue, (5) visual alike, (6) 
auditory alike, (7) begins same, (8) real word, and (9) 
self-corrects* A percentage score for each student was then 
computed for the number of miscues falling within each category* 

SRA (1979) achievement series test scores for vocabulary 
and comprehension were obtained from the students permanent 
school records* 
Design 

The interrater reliability of the scoring methods for the 
Opin procedure were evaluated by having independent raters score 
the students' initial responses to the moon passage* The 
independent raters also classified the students responses 
according to the category scoring methoc^ of the Opin procedure* 
The judgements in each case were then c rrelated using Pearson 
Product-Moment correlation procedures to assess the extent to 
which the scoring procedures are objective* 

Test-retest reliablity of the scoring methods designed for 
the Opin procedure were assessed by having the students complete 
the Opin procedure for the moon passage twice* The stability of 
the scores determined by each of the scoring methods was 
examined by comparing the scores the students received on their 
first opin sentence compeletion with the ones they received on 
their second try two weeks later* The comparisons were made 
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using Pearson product-moment correlation procedures. 

The same method of correlation was chosen to evaluate the 
relations between the Opin scores and standardized achievement 
test scores for vocabulary and comprehension. Pearson 
product-moment correlations were also used to ascertain the 
pattern of relationship between the scores on the Opin procedure 
and the miscue classification scores. 
Procedures 

All students selected to participate in the study were 
assembled in a common area within their school. The students 
were instructed to read the moon passage silently and complete 
the Opin sentence deletions. They were told that the purpose of 
the activity was to see whether or not students could figure out 
what the author might have written in each of the blanks. When 
the students completed the activity they were dismissed to 
return to their regular classroom. No information was given as 
to the actual details of the passage for any of the Opin 
deletions. All students finished the Opin procedure within 30 
minutes . 

During the next week, the same students were individually 
called to an office within the school and asked to read the 
earthquakes passage orally into a tape recorder « They were told 
to read the entire passage without asking for assistance, and to 
figure out wcrds, if necessary, by themselves. No indication 
was given as to whether or not they would be tested for 



ERIC 



Opin Procedure 11 



comprehension after reading this text segment. When the 
students finished reading the earthquakes passage they were 
allowed to return to their regular classrooms* All students 
completed the oral reading within 10 minutes. 

Two weeks from the time of the first session, all the 
students participating in the study were again assembled within 
a common area of their school. The students were presented with 
the moon passage for a second ti.ae and asked once more to finish 
the ends of the sentences that were left blank. Again they were 
told to try to figure out what the author might have said for 
each deletion. All students completed this activity within 30 
minutes. 

RESULTS 

Reliabilty 

In order to establish the interrater reliability of the 
Opin procedure, independent raters scored the responses of 37 
subjects to the moon passage. These judgements were then 
correlated to assess the extent to which the scoring procedure 
was objective. Table 1 presents the means and standard 
deviations for the Opin percentage scores. The Pearson 
product-moment correlation for these judgements was £ = .93 (p < 
.01). The interrater reliability was also assessed for the 
category scoring method. Table 2 presents the means and 
standard deviations for Opin classifications scores. Students 
were placed within one of the following classifications: (1) 
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fragmented^ (2) projective^ or (3) interactive, if 50% or more 
of their responses fell within one of the designated categories. 
The Pearson product-moment correlation comparing the category 
placement judgements of the two independent rater was found to 
be £ » .92 (p < .01). These results provide evidence that the 
scoring methods for the Opin procedure show sufficient 
interraier reliability to be used for informal assessment 
purposes. 

Test-retest reliabliity for the Opin procedure scoring 
method was assessed by comparing the performance of the 37 
subjects on the moon passage with a second retake performance on 
the same passage two weeks later. Table 1 displays the mean and 
standard deviation for the scores on the Opin retake. The 
Pearson product-moment correlation between the initial 
performance and the retahe performance on the moon passage was r 
= .47 (p < .01) r demonstrating a moderate but significant 
relationship for the scores on the two testing occasions. This 
modest relationship is not completely surprising considering the 
interpretative nature of the Opin procedure. 

Test-retest reliability was also calculated for the 
category scoring method. The results revealed a significant 
correlation of r » .71 (p < .01) for the category classification 
scores recorded for each of the two administrations of the moon 
passage. This test-retest relationship indicates that the 
category scoring method is relatively stable from one testing 
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session to the next# Taken together r these results suggest that 
the category scoring method is stable enough to allow for the 
diagnostic classification of students as Interactive, 
Projective, or Fragmented readers « 
Relationship to Standardized Achievement Measures 

Table 3 presents the means and standard deviations for the 
standardized measures of reading achievement, SRA vocabulary 
scores, and SRA comprehension scores* The correlation between 
the Opin Scores and the SRA vocabluary scores was £ » .SS (p < 
•01). For the SRA comprehension scores the correlation was 
found to be r = .61 (p < .01) • These moderate correlations 
between process and product measures of reading achievement are 
at the level that one would anticipate. They indicate the 
ability to interact with the meaning intended by the writer is 
associated with higher general reading ability. It also 
suggests that readers of higher reading achievement in general 
experience less distance when reading materials selected for 
their grade level. 
Relationship to Miscue Analysis 

The Opin procedure scores were compared with each of the 
miscue scores to find out whether the reader's propensity to 
interact with the author influences the kinds of miscues he or 
she makes when reading. Table 4 presents the means and standard 
deviations for the students' performances on each of the miscue 
measures. Pearson-Product moment correlations were computed 
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betweeen the students* percentage scores on the Opin procedure 
and the 12 miscue measures. The intercorrelations are presented 
in table 5. 

The highest significant correlationc between the Opin 
percentage scores and the Miscue measures occurred for Miking 
Sense { r = .51) r having the Same Meaning ( jc « •41) , and making 
Real Word substitutions ( £ « .56) • More modest correlations 

I- 

were found for Syntactic Fit ( £ » .32) r and Syntactic Fit Up To 
Miscue ( £ « .31) • All other correlations between the Opin and 
miscue scores were small and nonsignificant. In line with the 
predictions, these data suggest that there is an association 
between performance on the Opin procedure and the kinds of 
miscues readers are likely to make. More specifically, the 
positive correlations suggest that the closer the proximity 
between the reader and the author as measure by the Opin 
procedure the more likely the reader is to make miscues that 
preserve the author's meaning and make sense. 

The relations among the Opin Category Scores and the 
Miscues measures were also assessed using Pearson product-moment 
procedures. Table 5 presents the obtained intercorrelations. 
For category scores, significant although moderate correlations 
were found with Semantic Sense ( £ = .50), Syntactic Fit ( £ = 
.52) , and Syntax Fit Up To Miscue ( £ = .39). There was also a 
significant correlation with the use of Visual Clues ( £ s.43). 
No other relations between the Opin scores and the Miscue scores 
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were significants These results suggest that the classification 
of students as interactive ^ projective or fragmented readers is 
associated with the type of miscue they are likely to make while 
reading* It appears that the more readers attempt to interact 
with the material they read^ the more likely they are to make 
miscues that make sense and fit the syntax of the sentence ♦ 

DISCUSSION 

The results comfirm that the Opin sentence procedure has 
sufficient interrater reliability to be used as a means for 
assessing a reader's ability to interact with text during the 
act of silent reading. Both the percent score method and the 
category scoring method proved to, have adequate reliability 
for reading assessment purposes. The more modest test-retest 
correlations for the percentage scoring method suggests that 
readers change their responses to the opin deletions from one 
response session to the next. This is not very surprising in 
light of the open-nature of the response format. Upon a second 
reading y some readers might show improvement in their ability to 
interact with the text, while others might experiment with 
different answers or give less attention to the activity due to 
the fact that they had completed the task once before. The 
category scoring method demonstrated greater test-retest 
stability. The correlation was sufficient to suggest that most 
readers maintain the same basic approach ( interactive ^ 
projective, or fragmented ) when interacting with the text. 
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eventViough they do not produce exactly the same responses each 
time. This suggests that the Opin procedure may be useful for 
diagnosing habitual reading strategies. Current research is 
underway to investigate this possibility* 

The relations between the Opin procedure scores and 
traditional standardized measures of reading achievement suggest 
that proficient readers are more likely to attempt to interact 
with text when reading silently than less able readers. The 
pattern of correlations with the miscue scores also indicates 
that readers v/ho attempt to interact with the text while reading 
are more likely to make miscues that are consistent with their 
search for meaning. 

Taken together these results are consistent with theories of 
reading comprehension (Goodman^ 1967/1976; Rumelhart, 1984) that 
suggest that proficient readers integrate their previous 
experiences with the experiences intended by the author through 
the development of presuppositions, and use the details of the 
text to confirm, modify, or disconfirm these expectations. 

In summary, the Opin Procedure holds great promise for 
investigating how readers interact or fail to interact with 
authors. It appears that the Opin procedure assesses thinking 
process necessary to comprehension during silent reading. 
Moreover, the Opin procedure allows for the monitoring and 
assessment of comprehension during the act of reading with a 
minimal amount of interference. It is hoped that this study 
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will stimulate further investigation of the comprehension 
processes employed by readers during the act of silent reading* 
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Table 1 

Means and Standard Deviations for the Opin Percentage Scores 

Mean SD 

Moon Passage (Rater 1) 52.9 17.4 

Moon Passage (Rater 2) 53.3 17.3 

Moon Passage Retake 50.5 21.5 



ERIC 



21 



Opin Procedure 21 



Tabla 2 

Means and Standard Deviations for the Opin Category Scores 

Mean SD 

Moon passage (Rater 1) 2.24 .80 

Moon Passage (Rater 2) 2.14 .86 

Moon Passage Retake 2.14 .82 
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Table 3 

Means and Standard Deviations for the Achievement Test Scores 

Mean SD 
SRA vocabulary 6.7 3.4 

SRA Comprehension 6.7 4.0 
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Table 4 

Means and Standard Deviations for the Miscue Analysis Scores 





Mean 


SO 


Semantic Sense 


63.8 


15.8 


Same Meaning 


50.6 


17.4 


Syntactic Fit 


66.1 


16.6 


Syntax to T'.iscue 


87.2 


12.3 


Visual Similar 


75.7 


10.9 


Auditory Alike 


66.7 


13.5 


Begin Same 


71.4 


13.2 


real Word 


81.9 


11.8 


Self-Correct 


20.3 


12.5 
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Table 5 



Presents the Intercorrelations of 


the 


Opin and 


Miscue Scores 








A 


B 1 


2 


3 


4 


5 


6 


7 


8 


9 


A Opin 


.51 


.41 


.32 


.31 


-.13 


-.25 


-.22 


.56 


.15 


B Opin Category 


.50 


-.07 


.52 


.39 


.43 


-.05 


.03 


-.27 


-.15 


1 Makes Sense 




. 79 


. 66 


.45 


-.21 


-.05 


- . ^8 


.63 


. 12 


2 Meaning Same 






.50 


.22 


-.28 


-.18 


-.24 


.49 


.01 


3 Syntactic Fit 








.77 


.01 


.08 


-.08 


.39 


.09 


4 Fit up to Miscue 










-.04 


-.OJ 


-.23 


.33 


.41 


5 Visual Similar 












.68 


.72 


-.09 


.11 


6 Auditory Similar 














.55 


-.05 


-.01 


7 Begins Same 
















-.23 


-.19 


8 Real Word 


















.03 



9 Self-Correct 



n = 37 

* (P < .05) 
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Figure I ' 

1, Everyone knows the story of the volfaan. Khen the noon vas full* HE TURNED 

IWIO A MONSTER • And he did' ter- 
rible things. 

Cen the mooa really sake people aean end does the aoon resUy have soaethlng to do 

2. with the my people act? Two aclenticta h^rs at Misai's School of ICDICINE 



HAVE HADE A MEW DISCOVERY , Xhey found that the 

3. noon does seem to affect the vr.^ people behave. Dr. Arnold LlAer and Dr. CAROLYN 

SHERIN DID A STUDY . - checked 

the dates of mirder^ in thrKiaal area over a 15 year tlM period (1956-*70). Ihey fMnd 

4. that nany more murders took place during the FOLi: MOON OR THE MEtf MOON THAN AT 

OTHER TIMES 

Then th^ studied the wrders in the Cleveland (Ohio) area for a 13 year period 

5. (1956-70). Their findings were ABOUT THE SAHE 



. ilure murders took place at the time of the new or full 



moon. 

6. Why did the scientists CHOOSE TO MEASURE MURDERS 



? Hurders are violent acts. If the changes of the mooa have 



something to do with murders* they may have something to do with all violent acts. Does 

7. this mean we're all A LTTSIZ MOON MAD 

1 Ho. But it does suggest that the changes of the moon may have 



sonethlng to do with people's behavior « 



8. Scientists have known for a long time that the MOON PULLS ON THE WATERS OF 
THE EARTH The pull of the moon causes the 

9. water of the world.' s oceans to rise. This is what Is KNOWN AS TIDES 



. The pull of the moon is 



10. strongest at the time of the new or full noon. This stronger pull CAUSES HIGHER 

^^^^ . Scientists also know 

11. that the human body is mostly water. And aow this stu^ MAKES SOTIE SCTENTISTS 

WONDER . Could the pull of the noon 

change the waters of the .human bod^? And could such changes make certain people - or 

12 ♦ ALL mOPUB - MORE LIKELY TO BE VIOLENT _? 
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